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ABSTRACT 

We evaluate the performance of the SDSS DR8 redMaPPer photometric cluster catalog by comparing 
it to overlapping X-ray and SZ selected catalogs from the literature. We confirm the redMaPPer 
photometric redshifts are nearly unbiased ((Az) < 0.005), have low scatter (er 2 « 0.006 — 0.02, 
depending on redshift), and have a low catastrophic failure rate (« 1%). Both the Tx~X and M gas -A 
scaling relations are consistent with a mass scatter of 0"i nA m « 25%, albeit with a 1% outlier rate 
due to projection effects. This failure rate is somewhat lower than that expected for the full cluster 
sample, but is consistent with the additional selection effects introduced by our reliance on X-ray 
and SZ selected reference cluster samples. Where the redMaPPer DR8 catalog is volume limited 
(z < 0.35), the catalog is 100% complete above Tx > 3.5 keV, and Lx > 2 x 10 ergs -1 , decreasing 
to 90% completeness at L x ~ 10 43 ergs -1 . All rich (A > 100), low redshift (z < 0.25) redMaPPer 
clusters are X-ray detected in the ROSAT All Sky Survey (RASS), and 86% of the clusters are 
correctly centered. Compared to other SDSS photometric cluster catalogs, redMaPPer has the highest 
completeness and purity, and the best photometric redshift performance, though some algorithms do 
achieve comparable performance to redMaPPer in subsets of the above categories and/or in limited 
redshift ranges. The redMaPPer richness is clearly the one that best correlates with X-ray temperature 
and gas mass. Most algorithms (including redMaPPer) have very similar centering performance, with 
only one exception which performs worse. 
Subject headings: galaxies: clusters 



1. INTRODUCTION 



Galaxy c lusters are well known cosmological 
probes (e.g. IHenry et all I2009t IVikhlinin et all 12009b : 



Mantz et al.l 



Benson et al 




1Rc«o et al.N20injTcierc et al.l "l2012 : 
Hasselfield et al.1 120131 ). Indeed, 
galaxy clusters are a key component of the current 
efforts to probe our accelerating Universe with pho- 
tometric surveys such as the Dark Energy Survey 
(PES, [The PES Collaboration! 12003) . Pan-STARRS 
(|Kaiser et al.l |2002[ ). the Hyper-Suprime CamereQ, 
and the Large Synoptic Survey Telescope (LSST, 
ILSST Park Energy Science Collaborationl 120121 ). For 
these efforts to succeed, one requires a robust, well 
calibrated, fully optimized photometric cluster finding 
algorithm. 

This work is a companion paper to Rykoff et al. (2013, 
henceforth Paper I), which introduces redMaPPer, a new 
optical cluster finding algorithm specifically designed to 
take full advantage of these upcoming photometric sur- 
veys. The primary goal of this work is to evaluate the 
performance of redMaPPer against a variety of X-ray 
and SZ selected catalogs. The motivation for this type 
of comparison is manifold. 

First, while it is true that galaxy clusters were first dis- 
cover ed in the optical as far back as the 1800's (jBivianol 
120001 ) . and that the first la rge cluster catalogs relied 
on op tical observations (e.g. lAbelll 119581 : IZwickv et al.l 
H968t lAbell et al.lll989t) . ever since the 1990's photomet- 
ric cluster selection has been viewed as relatively unre- 
liable. In particular, optical selection is broadly con- 
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sidered to be prone to severe projection effects (jLucevi 
[19831 : iFrenk etalEmi . Ivan Haarlem et al.lll997t) . While 
these early results were concerned with photometric clus- 
ter finding with single band imaging, the modern lit- 
erature has clearly demonstrated that projection effects 
remain a critical concern for optically select ed samples, 
even with the advent of multi-band data (e . g.lCohn et al 



2T)07| :IWhite eHIMiWt iNoh fc Cohnl[20T lt LAngulo et al 



2012f ). While it has been argued that projection effects 
can be minimi zed through a ca reful treatment of pho- 
tometric data (lRozo et al.l 1201 ll Paper I), it remains to 
be explicitly demonstrated that a low incidence of pro- 
jection effects can be achieved. Because X-ray cluster 
selection is extremely robust to projection effects, a di- 
rect comparison of the redMaPPer catalog to X-ray se- 
lected systems and sources in the ROSAT All Sky Survey 
(|Voges et al.ll 19991) allows us to directly test the incidence 
of severe projection effects in redMaPPer. 

Second, it has long been recognized that photomet- 
ric mass proxies such as total galaxy counts and/or 
total cluster luminosity may not be very effective. 
That is, while these quantities are clearly correlated 
wit h cluster mass, they appear to exhibit large scat- 
ter dGladders et al .1 [2007t lRozo et al.ll2009at iSong et al.1 
However, recent work has argued tha t optimized 
redshi f t estimators can d eliver low scatter (jRozo et al.1 
I2009bt iRvkoff et al.|[20T2l) . although the arguments have 
been based on noisy data with an int rinsically large scat- 
ter (though see also I Andreonl I2IH2I ) . Here, we wish to 
explicitly test whether a scatter in mass at fixed rich- 
ness as low as has been claimed in the literature can be 
realized by comparing the redMaPPer richness estima- 
tor against low-scatter X-ray mass proxies such as X-ray 
temperature and gas mass. 
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Third, it is sometimes suggested that optically se- 
lected clusters represent a fundamentally distinct pop- 
ulation of galaxy clusters from X-ray and/or SZ se- 
lected galaxy clusters — the so-called X-ray under- 
luminous systems — as opposed to X-ray and opti- 
cally select ed cluster samples having regular scaling rela- 
tions (e.g.. iDonahue et alJl200U iRasmussen et al.ll20"06l 
Lopes et al.l 120091 [Dai et all l2010t lAndreon &: Morett 



20111: iBalogh et all !2Ullt IFlancJt Uollaborationl I^Ullc 
Wang et alJl2011b IRozo et al.ll2012d ). Of course, in de 



20111: iPlanck Collaborationl l2011d 



tail, no optical cluster catalog can be precisely the same 
as an X-ray or SZ selected catalog. Rather, the relevant 
question is whether the relative selection functions be- 
have as expected. For instance, optically selected clusters 
cannot be detected in X-rays if they are fainter than the 
flux limit of the X-ray survey, so one naturally expects 
the fraction of optically selected clusters detected in X- 
rays to decline in a simple, predictable manner with both 
richness and redshift. Is this, in fact, the case? Similarly, 
are there any massive X-ray and/or SZ selected clusters 
which one would expect to be detected in optical, but 
which are not? Addressing these questions if of critical 
importance in cementing photometric cluster selection as 
a viable alternative to X-ray and SZ cluster surveys. 

Of course, redMaPPer is far from the only clus- 
ter finding algorithm available in the literature to- 
day. At present, there exists a large vari- 
ety of photometric cluster finding algorithms, with 
roughly half of the cluster finders relying on pho- 
tome tric galaxy redshifts for cluster identification 
e.g..lKepner et al.|[T999Hvan Breukelen fc Clewlevl[2009l: 



Milkeraitis et alj|2010t iDurret et all 1201 lbl: [Szabo et alj 
201 U iSoares-Santos et all 120111: iWen et al.l 120121 ) and 



technique (lAnnis et a 


.1 119991: IGladders & Yeel I2000t 


Koester et al.l 12007a: 


Gladders et al.l 20071: iGal et al.l 


200S iThaniavur et al.ll2009t lHao et al. 2010). Given the 



variety of different algorithms and techniques employed 
in optical cluster finding, maximizing the utility of on- 
going and near future photometric surveys necessitates a 
detailed comparison of these various algorithms. In this 
way, we may adequately identify which techniques are 
better or worse suited to the specific question at hand, in 
particular cluster cosmology. The secondary goal of this 
work then is to compare and contrast the performance of 
redMaPPer against that of several other cluster finders. 

The basic analysis framework that we use for the 
comparison of redMaPPer to other cluster finding al- 
gorithms is fundamentally different than the type of 
cross-comparison work that has been performed in the 
past. Specifically, we do not directly compare redMaP- 
Per against the other catalogs. The reason is simple: 
consider, for instance, comparing two photometric clus- 
ter catalogs A and B. If the richness comparison of cat- 
alogs A and B is noisy, how can one tell whether A is 
at fault, or whether B is at fault? If a specific clus- 
ter is found in catalog A, but not catalog B, does that 
constitute a failure of catalog A, a failure of catalog B, 
or neither, or both? Because one does not know a pri- 
ori which catalog is "more correct" — and this needs to 
be properly defined — it is unclear what kind of infer- 
ences could be mad e from such a comparison (see e.g., 
iBahcall et al.l [20031 for further discussion). What one 
needs to perform a proper comparison then is some sort 



of "truth table", i.e., a definition of "more correct". 

In this work, we rely on X-ray and SZ selected cata- 
logs as a truth-table of sorts. So, for instance, given two 
optical catalogs A and B, we can unambiguously deter- 
mine whether the fraction of galaxy clusters detected in 
X-rays is higher for catalog A than for catalog B or vice 
versa. Provided galaxy clusters form a single population 
with well defined scaling relations, then a larger fraction 
of X-ray selected clusters can be unambiguously viewed 
as a good thing. Similarly, a lower scatter in X-ray tem- 
perature and/or gas mass at fixed richness is also clearly 
a good thing, as both of these quantities are recognized 
as very accurate mass proxies. 

In addition, it is our hope that as various features of 
photometric cluster finding algorithms are revealed to be 
more or less effective, that the field as a whole will con- 
verge into a more unified framework. As noted above, 
there are now at least 15 different cluster finding algo- 
rithms, all of which have different selection functions, and 
different richness and redshift estimators. Consequently, 
it is difficult to compare the results from various research 
groups, even more so given that each cluster finding algo- 
rithm is typically run on only one data set. By identifying 
which techniques are more or less effective, we hope the 
field can converge into a common language that will facil- 
itate communication. Conversely, if we find that several 
techniques arc all found equally effective, then there is 
a strong motivation to pursue multiple cluster detection 
avenues in future work, so as to provide an explicit test 
of cluster cosmology with several independent samples. 

Our work is laid out as follows. In Section [5J we intro- 
duce the redMaPPer catalog, and the various X-ray and 
SZ catalogs that will be employed in our analysis of the 
performance of redMaPPer. Section [3] discusses how we 
match galaxy clusters across catalogs, a process that is 
the fundamental stepping stone for all subsequent work. 
Section @] compares the redMaPPer redshifts to those of 
our reference catalogs, while Section [5] explore the X- 
ray and SZ scaling relations with the redMaPPer opti- 
cal richness. Section [6] discusses the relative X-ray and 
SZ completeness and purity of the redMaPPer catalog, 
and section [7] evaluates the efficacy of the redMaPPer 
centering algorithm. Finally, in Section [S] we perform a 
simplified version of all of the above analyzes for all pho- 
tometric cluster catalogs available in the SDSS (including 
redMaPPer). While this analysis is less thorough than 
that used for the detailed characterization of redMaP- 
Per — in particular we do not perform extensive visual 
inspection of all cluster catalogs and their matches to 
the X-ray and SZ catalogs — the analysis is completely 
homogeneous: all cluster catalogs are treated in exactly 
the same way. Section [9] summarizes our results and 
present our conclusions. When necessary, distances are 
estimated assuming a fiat ACDM model with Vt m = 0.27, 
and h = 0.7 Mpc. 

2. CATALOGS 

2.1. The SDSS DR8 redMaPPer Catalog 

The red-sequence Matched-filter Probabilistic 
Percolation (redMaPPer) algorithm (Paper I: Rykoff 
et al. 2013) is a photometric cluster findin g algorithm 
based on the optimized ri chness estimator A (jRozo et al.1 
I2009bt iRvkoff et al.l [20121 ) . redMaPPer identifies galaxy 
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clusters as overdensities of red-sequence galaxies. It 
relies on an iterative self-training technique to fully 
characterize the evolution of the red sequence as a func- 
tion of redshift, including zero-point, tilt, and scatter. 
As the algorithm utilizes all colors (it — g, g — r, r — i, 
and i — z) simultaneously, the "scatter" is characterized 
by a full covariance matrix. The algorithm then uses 
the resulting red sequence model, combined with simple 
radial and luminosity filters, to estimate the probability 
that any given galaxy belongs to any given cluster. The 
cluster richness A is the sum of these probabilities. In 
addition, the algorithm estimates cluster photometric 
rcdshifts by simultaneously fitting all high probability 
cluster members with a single red sequence model. 

The redMaPPer catalog used in this work is that ob- 
taine d from running the algorithm on the SDSS DR8 
data (jAihara et al.ll2011l ). The survey mask is the same 
as that used for the Baryon Acoustic Oscillatio n Sur- 
vey (BOSS) target selection ([Dawson et al.l [20131) . sup- 
plemented by some additional cuts around unmasked 
bright stars and foreground galaxies (e.g. M31), for a 
total area of « 10,000 deg . All richness estimates are 
corrected for masked area due to survey edges, bright 
stars, and bad fields, and the catalog is further trimmed 
so that no cluster is masked by more than 20%. Spec- 
troscopic rcdshifts used for photometric redshift training 
and v alidation are derived from a compilation of SDSS 
main (|Strauss et al.l I2002T ) . luminous r ed galaxy (LRG , 
lEisenstein et al.ll2001l) . and BOSS DR9 (lAhn et al.ll2012l) 
galaxy samples. For a detailed description of how the 
photometric and spectroscopic catalogs are used in the 
training and construction of the catalog, we refer the 
reader to Paper I. 

2.2. X-ray and SZ Catalogs 

The SDSS DR8 redMaPPer catalog will be compared 
to the X-ray and SZ catalogs described below. In all 
cases, we adopt a cut on data quality: if X is the cluster 
observable of interest (e.g. Lx, Tx, M gas ), we discard 
from the reference cluster catalog any systems with 
AX/X > 0.3. The catalogs employed in our analysis are: 



2001 . and CIZA (Ebcling et al.l [2001 iKocevski etaLI 



2007t) . while the contribu ting sere ndipitous catalogs ar e 



160D (IMullis et al.l[200l. 40 0D (IBurenin et al.l 120071) 



XCS : The XMM Cluster Survey (XCS: iMehrtens et~aH 
120121) is a serendipitous search for galaxy clusters using 
all publicly available data in the XMM-Newton Science 
Archive. The first data release is comprised of 503 
optically confirmed, serendipitously detected galaxy 
clusters. Of these, 261 have spectroscopic redshifts, and 
203 have photometric redshift estimates. We restrict 
ourselves to the sub-sample of 402 galaxy clusters with 
temperature estimates. By necessity, the X-ray temper- 
atures reported in the XCS catalog are not core-excised. 

MCXC: The Meta-Catalog of X-ray dete c ted C lus- 
ters of galaxies (MCXC: iPiffaretti etldl 120111) is 
a compilation of galaxy clusters based on pub- 
licly available X-ray data from both t he R OSAT 
All Sky Survey (RASS: iVoges et al.l Il999l ) and 
serendipitous searches in ROSAT pointed observa- 
tions. The RASS cont ributing ca talogs are NO- 



SHA RC lRomer et all l2000Tl. WAR PS dHorner eTatl 



RAS (iBohringer et al.ll2000[). RE FLEX (iBohringe: 
200 l . BCS (lEbeling et al. | ll998l) . SGP dCruddace et al 



retal 



2002J), NEP (jHenrv et al.l 120061) . MACS ([Ebeling et al 



and EMSS ([Gioia et alJfT^OT ). The data from 
each of the individual galaxy catalogs was collected 
and homogenized, deleting duplicate entries, and en- 
forcing a consistent X-ray luminosity definition. In 
the catalog, Lx is defined to be the X-ray luminosity 
in the 0.1-2.4 keV band within an i?sooc aperture. 
Unfortunately, the catalog does not include errors in the 
X-ray luminosity estimates. 

ACCEPT: The ACCEPT cluster catalog is a com- 
pilation of X-ray clu sters with deep Chandra data 
([Cavagnolo et al 1 I2009T ). All data were independently re- 
duced and homogeneously analyzed, with projected tem- 
perature radial profiles available for all galaxy clusters. 
However, a single core-excised X-ray temperature is not 
reported. We utilize the temperature and gas profiles 
to compute a spectroscopic-like c ore-excised temperatur e 
using the weighting scheme in iMazzotta et aLl (I2004D . 
The necessary integrals from IMazzotta et al.l ([20041 ) are 
discretized into a sum over the observed radial bins, and 
core-excision is done using a 150 kpc aperture when pos- 
sible. If no radial bins falls entirely outside this region, 
we do not perform core-excision. 

Errors in the X-ray temperatures are estimated via di- 
rect Monte Carlo: the density and temperature profiles 
are randomly sampled based on their reported errors, and 
these are used to compute the average temperature as de- 
scribed above. The error is defined to be the standard de- 
viation of the Monte Carlo samples for each cluster. Be- 
cause ACCEPT reports their interpolated temperature 
profiles (so as to match their gas density profiles) there 
is significant covariance between neighboring radial bins. 
This leads to an under-estimate of the true temperature 
uncertainties in our Monte Carlo method. Unfortunately, 
the non-interpolated profiles were not available, so we 
simply assume that 1-off neighboring bins are perfectly 
correlated, with no covariance between non- neighboring 
bins. This increases our estimated error by a factor of 
though we hasten to add that these errors have a 
negligible impact on the recovered scatter in X-ray tem- 
perature at fixed richness. 

There is one cluster in this catalog that deserves 
special mention. Specifically, visual inspection of the 
density and temperature profiles of Abell 1942 reveals 
an obvious failure in the automated data reduction 
for R > 500 kpc, so including the information in the 
reported profiles beyond these radius severely biases 
the recovered X-ray temperature. Consequently, when 
estimating Tx for this system, we truncate all profiles 
at R = 400 kpc. We have explicitly verified with 
the ACCEPT team that truncating the integration at 
R = 400 kpc for this galaxy cluster is appropriate, and 
that failure to do so will introduce large systematic 
errors in the recovered X-ray temperature (M. Donahue 
2012, private communication). We note that if we do 
not truncate the profiles at this radius for Abell 1942, 
then this cluster becomes a gross outlier in the Tx~ A 
relation (see section [3]). 
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Mantz: The Mantz cluster sample (jMantz et al.ll2010bl) 

is comprised of galaxy clusters contain ed within the 
ROSA T Brightest Cluster Sample (BCS: lEbeling et afl 
[Ml), the ROSAT-ESO F l ux-Lim ited X-ray sample 
(REFLEX: IBohringer et~aT1 [200l . and the bright 
sub-sa mple of the Massi ve Cluster Survey (Bright 
MACS: lEbeling et alJ[20loh . Clusters from each of these 
catalogs were selected by applying a redshift-dependent 
flux cut, so as to select the most luminous X-ray clusters 
in the various samples. A sub-sample of these galaxy 
clusters was observed with Chandra in order to derive 
X-ray temperatures and gas masses. Our analysis is 
restricted to this sub-sample of galaxy clusters with gas 
mass measurements. 

Planck ESZ: The Pla nck all-sky Early Sunyaev - 
Zcldovich Cluster sample (jPlanck Collaboration! l2011a| ) 
comprises 189 galaxy clusters detected with signal- 
to-noise S/N > 6 in the Planck early data release. 
Of these, 20 were newly detected clusters, most with 
perturbed morphologies in the X-rays. The clusters are 
typically massive low redshift clusters, with the effective 
mass threshold increasing with increasing redshift. 

ROSAT Bright an d Faint Source Catalogs: 

The ROSAT Br ight (jVoges et al.l [l999h and Faint 
(|Voges et al.ll2000h Source Catalogs are comprised of all 
X-ray sources in the ROSAT All Sky Survey. X-ray de- 
tection is based on count rate in the 0.1-2.4 keV band, 
but sources must also pass likelihood thresholds, and 
have a minimum number of photon counts. The resulting 
catalog has over 10 5 X-ray sources over the entire sky. 

2.3. Richness and Photometric Redshift Estimates of 
Clusters in the Reference Catalogs 

To facilitate cross-catalog matching, we explicitly mea- 
sure the richness and photometric redshift of every clus- 
ter in our reference catalogs that falls within our DR8 
galaxy mask. In doing so, we hold the center of the 
galaxy clusters fixed to the reported X-ray centers. We 
estimate the cluster richness A in exactly the same way 
as we do for clusters detected in the redMaPPer cata- 
log, as described in Paper I. We note that while we hold 
the cluster center fixed, when estimating cluster richness 
we do not rely on the reported cluster redshifts in the 
reference catalogs, but rather estimate the photometric 
redshift z\ from the photometric data. When measuring 
richness, the reported cluster redshift is only employed 
when initializing our iterative photometric redshift esti- 
mate algorithm. 

3. CLUSTER MATCHING 

Throughout this work, we will be comparing the prop- 
erties of the X-ray/SZ-selected cluster catalogs to the 
redMaPPer clusters, so we must first define an algorithm 
for matching galaxy clusters between different cluster 
catalogs. We consider two types of matching algorithms, 
namely cylindrical (or proximity) matching, and mem- 
bership matching. 

3.1. Cylindrical Matching 

Cylindrical matching is simple: given a cluster x G 
X, we wish to find the corresponding cluster match y S 



Y. We rank order all clusters x S X, and then find all 
clusters y within some physical radius i? max (evaluated 
at the redshift z x of cluster x) and within some maximum 
redshift offset Az max . Here, we set i? m ax = 1-5 Mpc and 
Az max = 0.1, and the redshift is always that reported in 
the original catalogs (as opposed to our own photometric 
redshift estimate). If more than 2 clusters y <G Y fall 
within the cylinder centered on x, we take the largest of 
the 2 as the correct match, and remove this cluster from 
consideration when matching subsequent clusters in X. 
Thus, all cluster matches are unique. 

The above procedure defines the cluster match y of a 
cluster x. Using a similar algorithm, we can then find 
the cluster match x of the cluster y, which may or may 
not be the same as the original cluster x. Matches are 
considered spurious if x ^ x, and are dropped from the 
matched cluster list. 

3.2. Membership Matching 

Membership matching follows the same general algo- 
rithm to enforce unique, two-way matches between the 
X and Y cluster catalogs — i.e. we first match Y to X 
and then X to Y, keeping only clusters which are two- 
way matches. The main difference is the criterion used 
to match a cluster y to a cluster x. Specifically, let pf be 
the probability that galaxy i belongs to cluster x € X, 
and p\ be the probability that galaxy i belongs to cluster 
y £ Y. We define the matching strength s(x,y) between 
two clusters x and y via 

s(x,y) = J2PiP V i- (1) 

i 

The match y to cluster x is that which maximizes the 
function s(x,y) at fixed x. Likewise, the cluster match 
x to cluster y is that which maximizes s{x,y) at fixed y. 
As before, if x ^ x then the match is considered spurious 
and dropped from the list of matching clusters. 

3.3. Testing The Matching Algorithms 

For the vast majority of our clusters, the results from 
the cylindrical and membership matching agree. For each 
of our reference catalogs, we visually inspect all clus- 
ters for which the cylindrical and membership matches 
disagree. When possible, any such ambiguous cluster 
matchings are resolved based on our visual inspection. 
These disagreements arc relatively rare, occurring in 
< 5% of all clusters. Notes for each of these systems 
are collected in Appendix lAl 

Additionally, recall that we have estimated the cluster 
richness for every cluster in our reference cluster catalogs. 
One should expect that all such clusters that satisfy the 
redMaPPer richness cuts should be matched to an exist- 
ing cluster in the redMaPPer catalog. We find that there 
are a total of 9 galaxy clusters across all catalogs which 
were unmatched. None of these are unmatched due to a 
failure of the matching algorithms. 

Specifically, one cluster is unmatched because it falls 
within an unmasked region in the SDSS with bad pho- 
tometry. Three of the remaining eight are unmatched 
because they formally fall outside the redMaPPer an- 
gular selection threshold — i.e. their location on the 
sky is such that more than 20% of the cluster is masked 
out. The remaining five galaxy clusters have richnesses 
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that are very close to the richness threshold applied, and 
therefore scatter in and out of the sample due to the small 
differences in redshift and / or cluster centers between the 
redMaPPer and reference catalogs. 

It is worth noting that these results also allow us to 
estimate the optical completeness, i.e. the fraction of 
galaxy clusters that pass our selection threshold but are 
not in the catalog due to catastrophic failures in the op- 
tical data and/or algorithm. Since there are 372 clusters 
MCXC satisfying our cuts, only one of which is rightfully 
unmatched due to bad photometry, we conclude that our 
optical incompleteness is < 0.3%. Notes on each of the 
nine clusters noted above are collected in Appendix [Bj 

4. REDSHIFT COMPARISONS 

We begin by comparing the redMaPPer photomet- 
ric redshift ("photo-z") estimator (z\) to the redshifts 
quoted in the reference catalogs. This is a necessary first 
step; all cluster observables derived using an incorrect 
redshift will be systematically biased, and will therefore 
compromise the study of cluster scaling relations, com- 
pleteness, purity, etc. Consequently, when comparing 
redMaPPer to other catalogs, our first task must be to 
identify catastrophic redshift outliers. 

For each matched cluster in our reference catalogs, we 
calculate the redshift offset Az = z\ — z lc f of each cluster. 
For the photometric redshift z\, we consider both the 
photometric redshift estimate obtained while holding the 
cluster center of the reference cluster fixed (see section 
12. 3|) . and the cluster redshift of the redMaPPer cluster 
match. 

The clusters are binned in redshift (based on our pho- 
tometric redshift), and we require the bins be sufficiently 
wide to contain at least 25 clusters per bin. We focus on 
the redshift span z £ [0.1,0.5] over which the redMaPPer 
richness estimates are most robust. At z > 0.5, richness 
errors become very large due to the extrapolation in lu- 
minosity between the SDSS depth and the luminosity cut 
of 0.2L* used to define A. 

As shown in Figure [TJ the resulting distribution of red- 
shift offsets typically exhibits a Gaussian core with pos- 
sibly a few outliers. The parameters for the Gaussian 
core are estimated using the median redshift offset for 
the mean, and 1.4826 x MAD for the standard devia- 
tion, where MAD is the median absolute deviation of 
the sample about the median^ All 4cr outliers arc iden- 
tified and discussed. An example of the distribution of 
redshift offsets for the MCXC catalog, along with the 
recovered Gaussian fit and 4cr cuts is shown in Figure [TJ 

Our results are summarized in Table [T] The photo- z 
performance is consistent with that quoted in Paper I, 
that is, our redshifts are unbiased at the < 0.005 level, 
with scatter that is « 0.008 for z < 0.3, but increasing 
to w 0.02 byzsj 0.5. The redMaPPer photometric fail- 
ure rate is small, with two redshift failures for the full 
MCXC cluster catalog, and two clusters with bad pho- 
tometry. The corresponding total failure rate (including 
bad photometry) is w 1.2% ± 0.6%. The photo-z per- 
formance is largely insensitive to the choice of cluster 
center, though differences can arise if cluster centers are 
separated by large (R ~ 1 Mpc) offsets. Comments for 

3 The factor 1.4826 relates the MAD to the standard deviation 
for a Gaussian distribution. 



each of the galaxy clusters identified as outliers can be 
found in Appendix [C] 

5. X-RAY AND SZ SCALING RELATIONS 
5.1. Methods 

We now consider the efficacy of the redMaPPer rich- 
ness A as a mass tracer by looking at cluster scaling re- 
lations. To decouple this analysis from the performance 
of our optical centering algorithm, we rely on the clus- 
ter richness measurements at the reported cluster center 
in the reference catalogs. Cluster centering will be ad- 
dressed in Section \7\ Because both richness and X-ray 
observables depend on redshift, we remove all redshift 
outliers from this analysis. These are dominated by er- 
roneous redshifts in the reference catalogs, but include 
four redMaPPer clusters. Note that the richness mea- 
surements are evaluated at our estimated photometric 
redshift, and not the reported cluster redshifts of the 
reference catalogs, so noise associated with photometric 
redshift uncertainties are included in our analysis. Fi- 
nally, we impose two data cuts: the first is that the clus- 
ter richness must pass the redMaPPer selection criteria, 
and the second is that the error in the external observable 
(Lx, Tx, etc) must be less than 30%. The latter cut im- 
pacts primarily XCS galaxy clusters with relatively few 
photon counts. 

The mean and scatter of the X-ray scaling relations 
are estimated us i ng a stand ard Bayesian fitter (e.g., 
IRozo et"aLl I2012at IKellvl 120071 ) with a uniform prior on 
the amplitude, slope, and variance of the relation. These 
fits are performed in redshift bins, which are chosen to 
match those in Section [3J Errors are estimated via boot- 
strap resampling. 

To check for possible systematic failures of redMaP- 
Per, and/or identify unique galaxy clusters, we have also 
implemented an automated outlier rejection criteria. We 
reject from our fit all 3cr outliers in the scaling relation. 
There are only two such outliers, which are discussed 
below. For now, we simply note that we believe that 
exclusion of these clusters from our fits is well justified. 

Of particular interest to us are estimates of the scatter 
in mass at fixed richness, which can be inferred from 
the scatter in X-ray/SZ obser yables at fixed richn ess as 
laid out in the Appendix of IRozo et all (|2012bD . For 
simplicity, we assume zero intrinsic covariance between 
the various X-ray and SZ observables and the cluster 
richness. Setting the correlation coefficient r between A 
and the ad d itional observable X to r = in Eq. A13 of 
IRozo et all (|2012bh . we find that the scatter in mass at 
fixed richness is given by 

o 

~2 _ X \ X 2 / \ 

a M\\ — ~2 a M\X \ L ) 

a X\M 

where olx\m is the slope of the observable-mass relation 
for the reference catalog. If r ^ 0, the scatter can be 
larger or smaller depending on the sign of r. For esti- 
mating the mass scatter, we adopt a fiducial scatter in 
mass at fixed M gas /T x /L x of 10%/15%/25% and slopes 
of a = 1.1, 1.5, and 1.6. These numbers are characteris- 
tic of the vast literature on cluster scaling r e lations (e.g. 
Vikhlinin et all l2009at iMantz et al.l 120 lObt iPratt et al.l 
2009t iMahdavi et al.ll2012t IRozo et al.ll2012ci and many 
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Figure 1. Left panel: Comparison between the MCXC redshifts and the redshifts of the matching redMaPPcr clusters. Labeled clusters 
are flagged as outliers, and are dominated by erroneous redshifts in the MCXC catalog (see Appendix [Cj. Right panel: Distribution of 
Az = z\ — 2MCXC m the redshift bin z\ £ [0.2,0.3]. z\ is either the photometric redshift at the reported center (black histogram) or the 
redshift of the matching redMaPPcr cluster (blue dashed histogram). The apparent left-right offset of the two histograms arises because we 
have utilized two slightly different binning schemes to help differentiate between the two histograms. The red curve is the best fit Gaussian 
to the black histogram, and the dashed red-lines mark the 4cr cut employed to identify outliers. 



Table 1 

Comparison of redMaPPer Photo- .zs to Reference Catalogs 



Catalog 



Redshift Bin (z\ — z IC {) 



Tot. CI. Bad z Ici Bad z x 



XCS: fixed center 
XCS: matched 



0.1,0.3 
0.3,0. 5 : 
0.1,0.3 
0.3,0.5 



0.0038 
0.0064 
0.0043 
0.0067 



0.0089 
0.0192 
0.0120 
0.0206 



il 

14 
41 
14 



MCXC: fixed center 


[0.1,0.2] 


0.0021 


0.0057 


163 


1 






[0.2,0.3] 


0.0037 


0.0089 


91 


3 






[0.3,0.5] 


0.0053 


0.0170 


75 


2 




MCXC: matched 


[0.1,0.2] 


0.0023 


0.0050 


163 


2 






[0.2,0.3] 


0.0041 


0.0083 


91 


3 






[0.3,0.5] 


0.0053 


0.0170 


75 


2 


2 



ACCEPT: fixed center 
ACCEPT: matched 


[0.1,0.5] 
[0.1,0.5] 


0.0055 
0.0050 


0.0094 
0.0080 


56 
56 


to to 




Mantz: fixed center 
Mantz: matched 


[0.1,0.3] 
[0.1,0.3] 


0.0043 
0.0031 


0.0067 
0.0057 


31 
31 




1 
1 


Planck ESZ: fixed center 
Planck ESZ: matched 


[0.1,0.5] 
[0.1,0.5] 


0.0030 
0.0025 


0.0092 
0.0081 


38 
38 


1 
1 





Note. — "Fixed center" means the photometric redshift was estimated while holding the 
cluster center fixed to that reported in the reference catalog. "Matched" means we compare the 
redshift in the reference catalog to that of the matching redMaPPcr cluster. a z is the width of 
the Gaussian core of the P(Az) distribution. "Tot. CI." is the total number of clusters in the 
redshift bin. "Bad z ro f" is the number of 4cr outliers due to erroneous redshifts in the reference 
catalog. "Bad ZyJ' is the number of erroneous redMaPPer redshifts. The statistics in this table 
do not include two redMaPPcr clusters that were compromised due to bad SDSS photometry. 



others). 

We emphasize that the scaling relations presented in 
this work have not been corrected for selection effects. 
This is because of two important considerations: first, 
our immediate goal is to quickly evaluate the perfor- 
mance of redMaPPer, leaving precise estimates of clus- 
ter scaling relations for future works (e.g. Greer et al, 
in preparation). Most importantly, though, the selec- 
tion functions of our reference catalogs are often either 



difficult to quantify without the tools employed in the 
construction of these catalogs (e.g. XCS, Planck ESZ), 
or downright impossible to quantify because the cata- 
logs are aggregates from multiple sources (e.g. MCXC 
and ACCEPT). For the remaining Mantz catalog, our 
reliance on M gas rather than Lx as the mass tracer im- 
plies that a proper treatment needs to account for the 
obvious covariance between these two X-ray observables, 
an analysis which is beyond the scope of this work. 
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In this context, it is important to note that systematic 
offsets in our recovered scaling relations scale with the 
variance of the relation. Thus, as long as this scatter 
is small, our recovered relations should be relatively ro- 
bust to selection effects. As we will see below, both the 
Tx~X and M gas -A scaling relations have small scatter 
(pa 20% — 23%), so we expect these relations to be rela- 
tively unbiased. The same cannot be said of the Lx~\ 
relation, which exhibits a very large scatter (pa 70%). 

5.2. Results 

Our results are summarized in Table [5] We again cau- 
tion that we expect the L^-A relation to be subject to 
large corrections from selection effe c ts. A proper treat- 
ment can be found in iRvkoff et al.l (|2012t) . We do not 
recommend drawing any conclusions from the Lx~M re- 
lation quoted here; we have included this information for 
completeness purposes only, and we will not discuss it 
further. 

The left panel in Figure [2] shows the Tx— A relation as 
probed by the XCS and ACCEPT cluster catalogs. The 
slopes and scatters of the two relations are consistent 
with each other, but their amplitudes are not. The am- 
plitude offset at the geometric mean of the pivot point 
of the two samples is 0.35. That is, the two data sets ex- 
hibit a sa 40% systematic offset in X-ray temperatures. 
This is unusually large, but there are many differences 
between the two X-ray temperature definitions, includ- 
ing the fact that the ACCEPT temperatures arc core- 
excised whereas the XCS temperatures are not. Conse- 
quently, we are not particularly concerned with the XCS- 
ACCEPT offset. Rather, the most significant result from 
this comparison is that the scatter in mass at fixed rich- 
ness inferred from the two samples are consistent with 
each other. We compute the inverse variance weighted 
mean of the inferred mass scatter from the XCS and AC- 
CEPT catalogs to arrive at er ln M | A = 0.26 ± 0.03. 

We can compare these results to those derived from the 
M gas -\ relation as probed by the Mantz galaxy clusters, 
shown in the right panel of Figure [2] The inferred mass 
scatter from this sample is <J\ n M\\ = 0.21±0.04, also con- 
sistent with the XCS and ACCEPT values. The inverse 
variance weighted mean from the M gas and Tx analysis 
is cinMU = 0.24 ± 0.02. We expect selection effects can 
impact the scatter by pa ±0.05, so cr lnA m = 0.24 ± 0.05 
is a more reasonable estimate of the scatter in mass at 
fixed richness. 

These resu l ts are in excellent agreement with those of 
IRvkoff et alj ([2012). Relative to that work, our analysis 
benefits from our reliance on low-scatter mass proxies, al- 
beit at the expense of introducing systematic uncertain- 
ties due to selection effects. In this context, we note that 
while the inferred mass scatter from the Lx~X relation 
in this work appears to be larger than that inferred from 
the TV—A and M gas -A relations, we do not consider this 
pr oblematic; The pro per analysis of the Lx~X relation 
in IRvkoff et al.l (|2012f) is consistent with a pa 25% scatter 
in mass at fixed richness. We note that our estimate for 
the scatter is comparable to that achieved by current S Z 
experiments (jBenson et al.ll20ll iHasselfield et al.ll20Tl . 

5.3. Richness Failures 

There are a total of two outliers over the full XCS, 
MCXC, ACCEPT, and MANTZ cluster samples. 



XCS J1203.8+0147: This cluster is the neighbor of a 
much brighter X-ray foreground cluster. A reanalysis 
of the cluster by the XCS collaboration has resulted in 
significantly larger uncertainties in the recovered temper- 
ature, with ATx/Tx = 0.46. This new error estimates 
disqualifies the cluster from our sample, and strongly 
suggests that deeper X-ray data and a detailed multi- 
source analysis that deblends the foreground cluster 
is required if this system is to be included in our analysis. 

40 0d J1416. 4+444 6- ' Th is cluster was initially reported 
in iVikhlinin et al.l (|1998| ). along with a spectroscopic 
rcdshift of z = 0.400. Visual inspection of the cluster 
reveals south-easterly and south-westerly extensions of 
the cluster galaxies, both of w hich are also appa rent in a 
recent weak lensing analysis ([Israel et al.ll2012T ). Based 
on SDSS spectra, the redshift of these components are 
z = 0.390 and z = 0.373, suggesting these are cluster 
super-positions contained within a larger super-cluster. 
There is also an additional more distant southerly struc- 
ture at z = 0.397. In short, this is a clear projection 
effect in the optical. 

It is difficult to estimate the failure rate in our rich- 
ness measurements from this data. The outlier fractions 
for XCS, ACCEPT, and Mantz galaxy clusters are 1/49, 
0/54, and 1/29 respectively, though only one of these fail- 
ures is due to a redMaPPer failure. This suggests a sa 1% 
failure rate due to projection effects in the redMaPPcr 
catalog. This failure rate is in addition to the sa 0.7% 
photometric redshift failure rate. 

5.4. SZ Comparison 

We now turn to a comparison of redMaPPer clusters 
to the Planck ESZ cluster catalog. In this comparison, 
we have opted not to attempt to constrain the Ysz _ A 
scaling relation directly. This is for two primary reasons. 
First, because of its broad beam, the Planck centering er- 
rors are typically very large, which can strongly bias the 
richness measurements at the Planck centers. Second, 
the Ysz values derived solely from the Planck early data 
are bo th biased and very noisy ijPlanck Collaboration! 
I2011bl ). Therefore, we leave a detailed analysis based 
on SZ follow-up of individual galaxy clusters to a future 
work (Greer et al, in preparation). 

In light of these difficulties, we have opted instead 
to check the completeness of the Planck ESZ cluster 
sample. Figure [3] shows the Planck signal-to-noise as 
a function of cluster richness for all clusters in the 
DR8 footprint, evaluated at the Planck center. Al- 
though the vast majority of the clusters are very rich 
(A > 70), there are two clusters that stand out as unusu- 
ally poor: PLCKG228. 5+53.1 (ZwCl 1023.3+1257) and 
PLCKG182.6+55.8 (Abell 963). Both clusters have obvi- 
ous central galaxies and both are strongly miscentered by 
Planck, with the radial offsets being 1.0 Mpc (6.4 arcmin) 
and 0.6 Mpc (3.0 arcmin) respectively. Both clusters are 
properly centered in the redMaPPer catalog, but even 
then their richnesses are relatively modest, A = 35.5 and 
A = 30.5 respectively. 

Given their low richness, the detection by Planck of 
these two galaxy clusters is somewhat surprising. Vi- 
sual inspection of the SDSS image of PLCKG228.5+53.1 
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Table 2 

X-ray Scaling Relations with Richness 



Catalog Redshift Bin A p ; vot 


Amplitude 


Slope 


Scatter 


Eq. Mass Scatter 


Outlier Fraction 


XCS 
XCS 
XCS 


0.1,0.3 
0.3,0.5 
0.1,0.5 


40.9 
49.7 
45.6 


1.129 ± 0.056 
1.283 ±0.071 
1.206 ± 0.044 


0.56 ±0.14 
0.57 ±0.15 
0.57 ±0.10 


0.194 ±0.055 
0.234 ± 0.062 
0.225 ± 0.042 


0.25 ±0.09 
0.32 ±0.10 
0.30 ± 0.07 


1/25 
0/24 
1/49 


MCXC 
MCXC 
MCXC 

MCXC 


0.1,0.2 
0.2,0.3 
0.3,0.5 

0.3,0.5 


48.9 
65.5 
62.9 

52.3 


0.289 ±0.053 
0.927 ± 0.086 
0.818 ±0.082 

0.478 ± 0.041 


1.23 ±0.12 

1.24 ±0.13 
1.57 ±0.11 

1.38 ±0.07 


0.66 ± 0.04 
0.77 ± 0.07 
0.64 ± 0.06 

0.70 ±0.03 


0.32 ±0.03 
0.41 ±0.05 
0.32 ±0.04 

0.36 ±0.02 


0/159 

0/85 

0/71 

0/326 


ACCEPT 


0.1,0.5 


94.5 


1.905 ± 0.032 


0.407 ± 0.066 


0.196 ±0.021 


0.253 ± 0.036 


0/54 


Mantz 


0.1,0.5 


106.0 


0.062 ± 0.052 


0.72 ±0.12 


0.212 ±0.032 


0.210 ± 0.040 


1/29 



Note. — Amplitude, slope, and scatter refer to the X-\ scaling relation, where X is the relevant X-ray observable, i.e. 
T x for XCS and ACCEPT, M gaa for Mantz, and L x for MCXC. Our convention is <lnX|A) = A + a ln(A/A pivot ) where 
A is the amplitude, a is the slope, and A p ; vot is taken to be the median cluster richness. "Eq. Mass Scatter" refers to 
the scatter in mass at fixed richness estimated based on the observed X-ray scaling relation as described in the text. We 
emphasize the Lx~A relation suffers from large systematic errors due to unmodeled selection effects. 
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Figure 2. Left panel: The Tjf— A relation for XCS (black points) and ACCEPT (red triangles) galaxy clusters. Errors not shown to 
avoid cluttering the plot. The bands show the uncertainty in the mean relation. We note the two data sets exhibit a f» 40% systematic 
offset in X-ray temperatures. This is due to several reasons, including the fact that the ACCEPT temperatures are core-excised while the 
XCS temperatures arc not. Right panel: The M gas — A relation for Mantz galaxy clusters. The solid line shows the mean relation, and the 
dashed lines shows the intrinsic scatter. The labeled galaxy clusters are gross outliers, and are discussed in the text. The inferred mass 
scatter from this sample is <J\ n M\\ = 0.21 ± 0.04. 



reveals a possible very high redshift galaxy cluster just 
North of the quoted Planck center (z p h oto = 0.68 ±0.04), 
which suggests that this detection may in fact be a high 
redshift system which was mistakenly associated with the 
low redshift cluster about 1.0 Mpc away. Deep optical 
follow-up of this field is desirable to either confirm or rule 
out this hypothesis. 

The second of the two clusters, Abell 963, is also part of 
the ACCEPT and Mantz cluster catalogs. This is a cool- 
core relaxed galaxy cluster, and is also a known strong- 
lensing system. Looking at the Tx~A relation from AC- 
CEPT, and at the M gas -A relation from the Mantz cata- 
log, we find that Abell 963 is both the hottest and largest 
M gas cluster relative to its richness in each of these two 
cluster samples. Given that the SZ signal is proportional 
to Tx x A/gas, it is not surprising that Abell 963 was 



detected by Planck despite its relatively low optical rich- 
ness. We have not explored whether this type of strong 
covariance between M g as and Tx is generic, or whether 
the high thermal pressure in Abell 963 is fortuitous. Note 
that even though Abell 963 is the hottest and most gas- 
rich cluster relative to its richness, it is not an outlier, 
being only 2a away from the expected Tx and Af gas val- 
ues. 

6. COMPLETENESS AND PURITY: 

6.1. X-ray and SZ Completeness 

The completeness of a cluster catalog may mean many 
different things depending on the context. As noted in 
Section [3J all except one of the reference galaxy clusters 
which satisfy the rcdMaPPcr selection threshold are de- 
tected by rcdMaPPcr, implying that the optical incom- 
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Figure 3. Sign al -to-noi se vs. richness for the 

IPlanck Collaboration (20lja|) cl uster sample. Richness is 

evaluated at the center reported in IPlanck Collaboration ( 2011a! '). 
which can be significantly offset from the true cluster center. The 
two highlighted clusters are unusually poor and are discussed in 
the text. 



pleteness — i.e., the fraction of objects that we should 
have detected but failed to so — is < 0.3%. 

In this section, we ask a different question: given all 
XCS and ACCEPT galaxy clusters with T x > T min , 
what fraction of these systems are included in the SDSS 
DR8 redMaPPcr cluster catalog? In addressing this 
question, we remove from consideration all redshift out- 
liers, both because this failure rate has already been char- 
acterized, and because redshift errors in the reference or 
redMaPPcr catalogs will compromise the relevant mass 
tracers (A, Tx, etc.). 

The left panel in Figure 0] shows the fraction of XCS 
and ACCEPT galaxy clusters found in the rcdMaP- 
Per catalog, as a function of the temperature threshold 
T m j n . As before, we limit ourselves to well measured 
clusters, with ATx/Tx > 0.3. At low redshifts, where 
the redMaPPer catalog is volume limited, we detect all 
X-ray clusters with Tx > 3.5 keV. For z > 0.35 the in- 
creasing detection threshold for redMaPPer systems nec- 
essarily increases the minimum temperature over which 
we achieve 100% completeness. Over the redshift bin 
z £ [0.3,0.5], this minimum temperature is Tx > 5 keV. 

The right panel in Figure [4] shows the fraction of 
MCXC galaxy clusters detected by redMaPPer as a 
function of the luminosity threshold Lx- While the 
z e [0.1,0.2] and z £ [0.3,0.5] bins suggest that redMaP- 
Per is 100% complete above Lx > 2 x 10 44 ergs _1 , 
we find two clusters in the z € [0.2,0.3] bin above 
this luminosity which were not matched. We visually 
inspected both of these systems: 

MCXC 31326. 2+1230: There is a very small (A « 11) 
group at the reported location. The cluster is also 
identified in MCXC as Abell 1735. However, the 
reported position is very distant from Abell 1735, which 
is easily detected by redMaPPer (A = 81.4). We assume 
the reported luminosity is appropriate for Abell 1735, 
in which case this cluster is in fact properly identified in 
redMaPPer. 



MCXC J0927. 1+5327: This cluster is clearly miscen- 
tered in the optical, which has caused the richness to 
fall below the detection threshold. Thus, this cluster 
is less a failure to detect than it is a case of cluster 
miscentcring. In Figure |4j the drop in completeness at 
Lx ~ 3.5 x 10 44 ergs/s is due to this cluster. 

In light of these findings, we have matched cluster 
MCXC J1326. 2+1230 to redMaPPer assuming the clus- 
ter should be at the location of Abell 1735. The z £ 
[0.2,0.3] curve plotted in Figure|4]includes this correction, 
and demonstrates that redMaPPcr is 100% complete for 
L x > 2 — 4 x 10 44 ergs _1 clusters. 

We have performed a similar analysis using the galaxy 
clusters in the Mantz and Planck ESZ catalogs. We find 
that redMaPPer is 100% complete with respects to both 
of these catalogs. One galaxy cluster deserves special 
mention though: cluster PLCKG96.9+52.5 is clearly a 
rich (A = 76.7) galaxy cluster that is not included in 
our SDSS redMaPPer cluster catalog. Nevertheless, we 
do not consider this galaxy cluster a redMaPPer failure. 
When centered at the Planck reported center, the SDSS 
galaxy mask is such that 17% of the galaxy cluster area 
is masked. However, this Planck cluster has a significant 
offset relative to the correct cluster center, which is obvi- 
ous upon visual inspection, and is correctly identified by 
the redMaPPer algorithm. At that location, the mask 
fraction for the cluster is > 20%, thereby failing to pass 
the redMaPPer cut. In other words, at the correct clus- 
ter center, cluster PLCKG96.9+52.5 does not fall within 
the angular mask used to define the redMaPPer catalog. 

6.2. Purity 

We now consider the converse of the question we ad- 
dressed section 16.11 what fraction of redMaPPer galaxy 
clusters are X-ray detected? We limit ourselves to X- 
ray detections rather than SZ detections because we can 
probe much lower masses by doing so. Moreover, we 
have opted to use the combined ROSAT Bright and Faint 
Source Catalogs as our proxy for an X-ray detected clus- 
ter. Because these catalogs do not require the detection 
of significant extended emission — as is required for X- 
ray cluster catalogs such as NORAS — this allows us 
to probe a lower detection threshold and to improve our 
statistics. 

We compute the fraction of redMaPPer clusters asso- 
ciated with RASS sources as a function of threshold rich- 
ness A and redshift. We bin the redMaPPer clusters in 
narrow redshift bins of width Az = ±0.025, starting at 
z £ [0.1,0.15] and extending up to z £ [0.45,0.5]. Within 
each redshift bin, RASS sources are associated with a 
redMaPPer cluster if the angular separation AO is less 
than 800 kpc at the median redshift of the clusters in the 
bin. We have found this radius is sufficiently large to 
capture most true associations while keeping the num- 
ber of false matches to a minimum. Our chosen aper- 
ture corresponds to angles ranging from 2.0 arcmin to 
5.4 arcmin depending on the redshift. The probability 
of a chance association for these apertures is « 10% at 
z = 0.1, falling quickly with redshift to < 2% chance 
associations by redshift z = 0.35. 

Figure [5] shows the results of this matching exercise. 
As we would expect, there is some richness A m ; n above 
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Figure 4. Left panel: Fraction of XCS or ACCEPT galaxy clusters detected in redMaPPer, as a function of the temperature threshold 
Tmi n , as labeled. At low z, redMaPPer is complete for Tx > 3.5 keV, with an increasing completeness threshold for z > 0.35. Right panel: 
Fraction of MCXC galaxy clusters detected in redMaPPer as a function of the luminosity threshold Lx , for three different redshift bins. 



which all redMaPPer clusters are X-ray detected, and 
this richness A m i n increases with redshift. More gener- 
ally, at fixed richness, the matched fraction decreases 
with increasing redshift. Note that even for our high- 
est redshift bin, z £ [0.45,0.5], « 30% of the A > 100 
clusters are X-ray detected. 

In our highest redshift bins, there is an obvious flat- 
tening of the matched fraction at low richness, a clear in- 
dication that those matches are spurious. Interestingly, 
this flattening occurs at a level that is higher than the 
matching rate for random points quoted above. Thus, 
the flattening of the curves appears to be impacted by 
the AGN rate in galaxy clusters. Note that the chance 
association rate is always much lower than unity, so the 
high detection rates for rich low redshift clusters are very 
clearly true physical associations. 

Looking at the left panel of Figure there is one red- 
shift bin in particular that looks peculiar, namely the 
z £ [0.3,0.35] redshift bin (light green). In this red- 
shift bin, the fraction of X-ray detected clusters drops 
at A > 110. Above this richness there are nine clus- 
ters, four of which are not detected in X-rays. 3/4 sys- 
tems that are not X-ray detected exhibit 2 clear distinct 
galaxy clumps, with each clump hosting a good central 
candidate. In all 3 cases, the candidate central chosen by 
redMaPPer is not matched to an X-ray source, but the 
alternate candidate central is, suggesting these systems 
are X-ray detected, but grossly miscentered. It is worth 
noting that in all 3 of these cases, redMaPPer selected 
the second candidate central as the second most likely 
central galaxy. Correcting for these three systems makes 
the z £ [0.3,0.35] curve much more consistent with the 
remaining redshift bins, but we have chosen not to apply 
this correction in Figure [5] 

We now investigate whether the evolution of the curves 
in the left panel of Figure [5] is consistent with the naive 
expectation that the redMaPPer galaxy clusters form a 
homogeneous cluster sample with a well defined Lx~ 
A scaling relation. Specifically, we hypothesize that 
the only reason why the fraction of galaxy clusters de- 



tected in X-rays decreases is due the difficulty of detect- 
ing low richness/high redshift clusters in RASS. Let us 
then define a richness proxy for X-ray flux. Assuming 
Lx oc Al 1 - 6 , and M oc A, we arrive at Lx oc A 16 . Ig- 
noring K-corrections and intrinsic evolution in the Lx~X 
relation, one expect that the X-ray flux of a galaxy clus- 
ter is simply proportional to 



F x oc X 1(i D L (zy 



(3) 



where Dl{z) is the luminosity distance. Thus, a cluster 
of richness A at redshift z has the same X-ray flux as a 
cluster of richness Aq at redshift zq where 



A = A 



n , \ \ 2.0/1.6 

Dl(z) 



(4) 



Assuming that X-ray detection is limited only by an ef- 
fective flux threshold, it follows that all the curves in the 
left panel of Figure [5] should scale onto each other if we 
scale the x-axis according to Eqn. |4] This is shown in the 
right panel of Figure [5l where we have set zq = 0.13 as 
appropriate for our lowest redshift bin. The good agree- 
ment between the various curves demonstrates that the 
X-ray detection rate is limited primarily by the effective 
flux limit of RASS. 

7. CENTERING 

One of the most difficult questions to address within 
the context of optical cluster finding concerns finding the 
center of galaxy clusters. While in many cases there 
is a clear dominant cD galaxy that can be adopted 
as the cluster center, often times there can be more 
than one such candidate central. This ambiguity can 
often be removed with the a ddition of high resolu- 



tion X-ray and/or SZ data (e.g.. iMenanteau et al 
Song et al.N2012t iStott et alJ l2012t iMahdavi et al 



2012 



2012 



von der Linden et al.ll2012i rRoughlv speaking, the cor 



rect central galaxy is the largest cD galaxy closest to the 
X-ray/SZ center. We emphasize, however, that having 
a good centering proxy is critical for the visual selection 
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Figure 5. Left panel: Fraction of redMaPPer galaxy clusters above a given richness A matched to a source in the combined ROSAT 
Bright and Faint Source Catalogs. Each of the curves corresponds to a different redshift bin, as labeled. redMaPPer clusters are matched 
to ROSAT sources if they fall within an angular aperture corresponding to 800 kpc at the cluster redshift, which ranges from 2.0 arcmin 
at the highest redshifts to 5.4 arcmin at the lowest redshifts. Right panel: As left panel, but the richness is scaled by distance so that the 
x-axis can be considered a flux proxy. We have ignored K-corrections and any intrinsic evolution in the Ljsf-A relation. The fact that the 
various curves line up demonstrates the X-ray detection rate roughly traces a constant flux limit, as expected. 



to be robust. We now investigate how often redMaPPer 
fails to select the correct central galaxy by evaluating 
its performance in galaxy clusters with high resolution 
X-ray data. 

We first perform visual inspection of the redMaPPer 
clusters in the the XCS, ACCEPT, and Mantz galaxy 
cluster catalogs to determine the correct central galaxy 
for each cluster. We restrict our analysis to this sub- 
sample of galaxy clusters because of the excellent X-ray 
ccntroiding that can be achieved with high resolution X- 
ray instruments. Ultimately, our decision of which galaxy 
is the correct cluster central is subjective, but proximity 
to the reported X-ray center was most often the primary 
criterion used to select the correct central galaxy: we 
would not attempt this analysis without the X-ray data. 
We ignore photometric redshift failures in our centering 
analysis, and we also remove any clusters where the X- 
ray gas clearly extends beyond the X-ray detector 
problem that affects some of the XCS systems. Notes on 
galaxy clusters that merit some discussion are collected 
in Appendix ITU 

The left panel in Figure [6] shows the offset distribution 
between the central galaxies chosen in our visual inspec- 
tion and the X-ray center reported in each of our refer- 
ence catalogs. It is obvious that there are two distinct 
cluster populations contributing to the overall distribu- 
tion. The first population accounts for w 80% of the 
X-ray clusters, and is comprised of X-ray clusters where 
the X-ray centroid and the position of the central galaxy 
are essentially in perfect agreement (R < 50 kpc). The 
remaining 20% of the galaxy clusters are merging sys- 
tems where the gas is significantly offset from the central 
galaxy, though it's very rare to find systems with an off- 
set larger than R w 300 kpc. 

We turn now to explore how often did redMaPPer 
choose the correct central galaxy. We combine all three 
reference catalogs into a single collection of 121 unique X- 
ray clusters with high resolution X-ray imaging. We find 



that the redMaPPer algorithm successfully recovers the 
central galaxy selected via visual inspection 86.0%±3.2% 
of the time. The right panel of Figure [6] shows the dis- 
tribution of radial offsets between our visually assigned 
cluster centers and the redMaPPer centers for the 14% 
of central galaxies that were not correctly selected. The 
figure legend also indicates the centering success rate for 
redMaPPer for each of the individual cluster catalogs. 
The offset distribution can be roughly approximated by 
a uniform distribution extending out to 0.8 Mpc (dashed 
green line). Better statistics are required to obtain a 
more accurate centering model, as there are only 17 clus- 
ters in this figure. 

8. COMPARISON TO OTHER OPTICAL 
CATALOGS 

We now turn our attention to a comparison of the 
performance of redMaPPer to that of other photomet- 
ric cluster finding algorithms that have been applied to 
SDSS data. The structure of this comparison follows 
closely the structure of the paper as a whole. As our goal 
in this section is to compare redMaPPer to other cata- 
logs, as opposed to providing a detailed characterization 
of all cluster catalogs, our analysis in this section is pur- 
posely less thorough than that presented previously for 
the redMaPPer catalog alone. Specifically, we rely solely 
on cylindrical matching (sec Section [3. 1| . and we do not 
take additional steps such as visual inspection of cluster 
matches to ensure the matchings are appropriate. In ad- 
dition, we do not take care to find all redshift outliers 
between each pair of cluster catalogs. These simplifica- 
tions necessarily have a small quantitative impact on the 
recovered statistics, but still allow us to fairly compare 
different cluster catalogs. In particular, in this section we 
recompute the redMaPPer statistics in exactly the same 
way as is done for the additional optical catalogs, and 
refer the reader to the previous sections for the robust 
statistics of the redMaPPer catalog performance. 



12 



Rozo & Rykoff 




Figure 6. Left Panel: Offset distribution between the central galaxy selected by visual inspection and the reported X-ray center for 
clusters with high resolution X-ray imaging. Right Panel: Distribution of radial offsets between our visually assigned cluster centers and 
the redMaPPer centers for the 14% of central galaxies that were not correctly selected. The redMaPPer centering success rate — i.e., the 
frequency with which our choice of central galaxy agreed with redMaPPer — is in the figure legend. The dashed dark-green line corresponds 
to a uniform miscentering model that extends to iJ ma x = 0.8 Mpc. 



8.1. Comparison Catalogs 

We compare the redMaPPer cluster catalog to four 
additional SDSS optical cluster catalogs. 

maxBCG: maxBCG is a red-sequence cluster finding 
algorithm, where cluster members are sele cted on the ba- 
sis of magnitude-independent color cuts (|Koester et al.l 
I2007bft . Clusters are centered at the brightest cluster 
member, and the richness is the total number of cluster 
galaxies above a luminosity threshold 0.4X*. The 
catalog spans 7500 deg 2 , and is limited to the redshift 
range z £ [0.1,0.3] (jKoester et al.ll2007al) . 

gmBCG: gmBCG is a generalization of maxBCG 
which does not rely on a pre-paramctcrized model 
for the red- sequence of gala xy clusters as a function 
of redshift (|Hao et al.l 12010ft . Instead, the algorithm 
decomposes the color distribution of galaxies in a 
given field into two Gaussian components using an 
error-corrected Gaussian Mixture Model (the "gm" 
in gmBCG). In fields where there is a galaxy cluster, 
one of the two gaussians in the GM decomposition 
is narrow. Galaxies that contribute to the narrow 
Gaussian component are identified as cluster members. 
Clusters are centered at the brightest cluster member, 
and the richness is the total number of cluster galaxies 
above a luminosity threshold 0.4L». The catalog spans 
ss 8,200 deg 2 across a redshift range z £ [0.1,0.55]. 

A MF: The Adapt ive Matched Filter cluster cata- 
log (jSzabo et al .11201 TT) is based on a maximum likelihood 
algorithm. In this sense, some of the philosophical under- 
pinning of the algorithm are similar to those of redMaP- 
Per, but there are some crucial differences. Specifically, 
unlike redMaPPer, the AMF algorithm does not rely on 
red-sequence galaxies. Rather, it utilizes all galaxies, re- 
lying on photometric redshift estimates to estimate clus- 
ter membership. In addition, the AMF centers galaxy 



clusters by maximizing the cluster likelihood over posi- 
tion in the sky, so the center of a galaxy cluster need 
not coincide with the brightest cluster galaxy. We also 
note that the AMF algorithm employs spectroscopic data 
where available in order to assign cluster redshifts. The 
cluster richness is the total luminosity of the clusters in 
units of L*. The catalog spans w 8,400 deg 2 across a 
redshift range z g [0.045,0.78]. 

Note: Because AMF relies on spectroscopic data 
where available, we remove clusters with spectroscopic 
redshifts from the sample when estimating the photo- 
metric redshift performance of AMF. 

WHL: The WHL cluster catalog (|Wen et al.l [20121) 
utilizes galaxy photometric redshifts and a Friends-of- 
Friends algorithm to group galaxies into distinct clus- 
ters. The cluster redshift is estimated using the median 
photometric cluster redshift of cluster members, and the 
richness is the total cluster luminosity in units of L*. 
The catalog spans w 14,000 deg 2 across a redshift range 
z e [0.05,0.78]. 

Note: As can be seen in Appendix [El WHL employs a 
spectroscopic redshift cut such that clusters with |z sp ec~ 
Zphotol < 0.055 are discarded from the cluster catalog. 
Consequently, we evaluate the redshift performance of 
the WHL catalog with spectra that are exclusive to DR9, 
which was not available at the time WHL was published. 
In addition, we note that WHL removed 3.6% of their 
clusters after visual inspection. When estimating the 
fraction of catastrophic photometric redshift outliers we 
present two results: one for the catalog as published, 
and the other where we assume that half of the visually 
removed clusters were redshift outliers. 

8.2. Comoving Densities and Data Homogenization 

One important difficulty in comparing different cluster 
catalogs is that one generically expects the performance 
of all cluster finders to degrade as one moves to lower 
richness systems. Consequently, a fair comparison must 
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Figure 7. Comoving density ciS el & function of redshift for galaxy 
clusters in each of the five optical cluster catalogs under consid- 
eration. A fair comparison of galaxy clusters requires a uniform 
selection threshold, which we approximate by demanding that all 
catalogs have the same redshift-dependent space density as the 
redMaPPer catalog. This is implemented using a redshift depen- 
dent richness cut. 

restrict itself to the "same" selection threshold. In prac- 
tice, each cluster finder uses a different richness and/or 
selection threshold definition, so there is no unique way 
of enforcing this condition. Here, we use the comoving 
density of the galaxy clusters as a proxy for selection 
threshold. 

Figure [7] shows the comoving density of galaxy clusters 
for each of the five optical cluster catalogs we consider. 
We see that redMaPPer contains fewer systems per unit 
volume than any other cluster catalog. As discussed in 
Paper I, this is by design: we have been purposely con- 
servative in our application of a selection threshold for 
the SDSS DR8 redMaPPer sample in order to ensure the 
highest possible data quality^ As the comoving density 
of the redMaPPer catalog is smaller than all the other 
cluster catalogs, we apply a redshift-dependent cut on the 
richness appropriate for each individual catalog such that 
the resulting cluster sub-samples match the redMaPPer 
comoving density as a function of redshift. 

8.3. Photometric Redshift Performance 

We evaluate the photometric redshift performance of 
the various catalogs by u sing the spectros copic redshift 
catalog from SDSS DR9 (|Ahn et all 120121 ) . Specifically, 
we compare the photometric redshift estimate of each 
galaxy cluster to the spectroscopic redshift of either the 
central galaxy (for catalogs with such a galaxy) or the 
brightest cluster galaxy (for AMF, which does not specify 
a central galaxy). 

For each sample of galaxy clusters we estimate four 
statistics: the photometric redshift bias, its standard de- 
viation, its skewness, and the fraction of 4c photomet- 
ric redshift outliers. These statistics are estimated as a 
function of redshift as follows. First, we select all clus- 
ters in a bin z ± 0.025. We compute the redshift offset 

4 For example, were we to relax the redMaPPer selection thresh- 
old to A = 5, the redMaPPer comoving density would be higher 
than that of all other cluster catalog by more than a factor of 2. 



Az = Zgpoc — Zphoto, and then estimate the median red- 
shift and the median absolute deviation of Az. We then 
select all galaxy clusters with 

\Az - mcd(Az)| < 4 x 1.4826 x MAD (5) 

where MAD is the median absolute deviation. We then 
use the standard cumulant-based (aka fc-statistics) esti- 
mator to determine the mean, variance, and skewness 
of this cluster sub-sample. The scatter a z is defined as 
the square-root of the estimated variance. Finally, we 
estimate the fraction of 4cr z outliers based on the calcu- 
lated mean and variance. As noted in section f8.lt when 
evaluating the AMF catalog we restrict our analysis to 
clusters with photometric redshifts only, and the WHL 
analysis is restricted to clusters with spectra exclusive to 
DR9. 

Figure |S] collects the four statistics computed above: 
the photometric redshift bias (top- left), scatter (top- 
right), skewness (bottom- left), and outlier fraction 
(bottom- right). The least biased algorithms are redMaP- 
Per and WHL, while redMaPPer and gmBCG have the 
smallest skewness. redMaPPer outperforms the remain- 
ing catalogs in terms of scatter and it has the lowest rate 
of catastrophic outliers. In short, redMaPPer has the 
best photometric redshift performance of the five cata- 
logs we considered. 

8.4. X-ray Mass Scatter 

We now compare the performance of the redMaPPer 
richness estimator to the richness estimators of other 
catalogs. To do do so, we match each optical catalog 
to our three high resolution X-ray reference catalogs — 
XCS, ACCEPT, and Mantz — using cylindrical match- 
ing with a radius of R = 1 Mpc and a redshift width 
of Az max = 0.05. For simplicity, we adopt a single red- 
shift bin of Zphoto € [0.1,0.5]. We use the same fitting 
algorithm as that employed in Section [5] to estimate the 
scaling relations, including our automated 3cr outlier re- 
jection. 

In this context, it is important to note that even if 
there was no correlation between a given X-ray observ- 
able and cluster richness, our test will always recover a 
finite scatter. For instance, the scatter in M gas at fixed 
richness cannot be larger than the rms in M gas in the 
Mantz cluster sample. Consequently, in order to mea- 
sure the efficacy of a richness estimator as a mass proxy, 
it is not sufficient to simply measure the scatter in X-ray 
observables at fixed richness; one must also test whether 
the observed scatter is significantly smaller than that ex- 
pected in the absence of any correlation. 

We estimate the significance of a sub-random scatter 
via Monte Carlos. For each optical and X-ray cluster 
catalog pair (e.g., redMaPPer and XCS), we randomly 
scramble the richness values among all the matched clus- 
ters and remeasure the X-ray observable-richness scatter 
as above. This procedure is repeated 1000 times to es- 
timate the uncertainty in the scatter for the "random 
richness" case. This uncertainty is added in quadrature 
to the error of the recovered scatter for the unshuffled 
catalog, and then used to evaluate the significance of the 
difference in scatter between the original and richness- 
shuffled catalogs. 

As a final test the efficacy of the various cluster richness 
estimators, we have also estimated the Pearson corrcla- 
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Figure 8. Top-Left: Photometric redshift bias as a function of redshift for each of the five optical catalogs under consideration. Top- 
Right: Photometric redshift scatter. Bottom-Left: Skewness of the photometric redshift offset distribution. Bottom-right: Fraction of 
4<r redshift outliers. The dashed red-line assumes half of the 3.6% catastrophic failures in the WHL catalog that were removed from visual 
inspection were redshift outliers (see text). 



tion coefficient for each of the cluster samples, limiting 
ourselves to clusters that are not flagged as outliers by 
our automated outlier rejection algorithm. 

Our results arc summarized in Table [3] We see that in 
all cases the redMaPPer richness estimator results in the 
smallest scatters. Furthermore, the reduction in scat- 
ter relative to the shufflcd-richncss catalog is also most 
significant in the case of redMaPPer catalog. In fact, 
redMaPPer is the only catalog that shows consistent ev- 
idence (> 3c) that the X-ray observable-richness scatter 
is significantly lower than that of a richness-shuffled cat- 
alog. Similarly, redMaPPer always exhibits the largest 
correlation coefficients across all X-ray catalogs. In short, 
the redMaPPer richness is clearly better correlated with 
X-ray temperature and M gas than the richness measures 
of the remaining optical catalogs. 

8.5. Completeness 

To estimate the X-ray completeness of a given opti- 
cal catalog, we must first determine whether any given 
X-ray cluster falls within the footprint of the optical clus- 



ter catalog in question. Unfortunately, we do not have 
the detection masks for each individual cluster catalog, 
so we must resort to an approximate method to make 
this decision. We assume that X-ray clusters that fall 
within 40 arcmin of any optical cluster are within the 
optical footprint of the corresponding cluster catalog. 
The choice of this aperture is motivated by the angu- 
lar density of redMaPPer clusters, which corresponds to 
an average of ~ 1 cluster per circle of radius 20 arcmin. 
The motivation for setting this angular scale based on 
redMaPPer systems is motivated by the fact that the 
redMaPPer catalog is the one with the lowest density of 
galaxy clusters. Of course, our method for determining 
which clusters fall insider or outside the various masks 
is not ideal, but it docs have the benefit of treating the 
footprint of all the optical catalogs in the same way. 

Having established which X-ray clusters fall within the 
footprint of a given optical cluster catalog, we select all 
X-ray systems with z € [0.1,0.5], except for when com- 
paring with the maxBCG catalog, in which case we re- 
strict ourselves to the redshift range z £ [0.1,0.3]. We 
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Table 3 

Comparison of the efficacy of different richness measures as mass tracers 



XCS ACCEPT Mantz 

Catalog Scatter Sig. r Scatter Sig. r Scatter Sig. r 

redMaPPer 0.244 ± 0.035 3.0<r 0.61 0.200 ±0.021 4.1a 0.67 0.201 ± 0.033 6.4<r 0.83 

maxBCG 0.430 ±0.132 0.7a 0.40 0.326 ±0.122 1.6a 0.54 0.363 ± 0.064 I.Oct 0.43 

gmBCG 0.275 ±0.060 1.5<r 0.56 0.203 ±0.024 3.5cr 0.66 0.357 ± 0.052 2.3cr 0.63 

AMF 0.273 ±0.042 1.8cr 0.53 0.222 ±0.037 1.4ct 0.53 0.227 ± 0.094 2.7cr 0.79 

WHL 0.305 ±0.034 2.3cr 0.52 0.233 ±0.020 1.3ct 0.39 0.335 ± 0.058 3. Oct 0.70 

Note. — The XCS/ACCEPT scatter is the scatter in Tx at fixed richness, while Mantz scatter is 
the scatter in M gas at fixed richness. In all cases, "Sig." is the significance of the reduction in scatter 
relative to the case where the cluster richnesses are randomly shuffled, while r is the Pearson correlation 
coefficient. 



then use the cylindrical matching method of Section IQ1 
(with radius 1 Mpc and Az max = 0.05) to obtain a set of 
exclusive two-way matches. As in other sections, when 
evaluating the completeness we enforce density matching 
with the redMaPPer catalog. 

Figure [9] summarizes our completeness results. When 
computing the completeness as a function of Tx (left 
panel), we compute the completeness function using the 
XCS and ACCEPT catalogs, and take the average of 
the two completeness functions. When computing the 
completeness as a function of Lx (right panel), the lu- 
minosities are estimated using the MCXC catalog. We 
observe that in both cases the redMaPPer catalog is the 
most complete. We emphasize that while this analysis 
has the virtue of treating all optical cluster catalogs iden- 
tically, the resulting completeness functions are underes- 
timated due to the approximate handling of the mask, 
the bad redshifts in the X-ray catalogs, and the lack of 
optical inspection to adjust spurious matches. For the 
true redMaPPer completeness function, see Figure |4j 

8.6. Purity 

Following Section 16. 2[ we estimate the purity of each 
photometric cluster catalog by matching it to the ROSAT 
Bright and Faint Source Catalogs. We use four redshift 
bins from z = 0.1 to z — 0.5 in steps of Az = 0.1. X-ray 
sources are considered matched to galaxy clusters if they 
are within an angular separation of 1 Mpc at the median 
redshift of the appropriate redshift bin. 

Our results are summarized in Figure [TOl Because the 
X-ray detection rate is a function of richness, we must 
properly account for the fact that the different cluster 
catalogs all have different richness definitions. Therefore, 
we present our results as a function of density rather than 
richness. Each panel corresponds to a different redshift 
bin, and each color corresponds to a different cluster cat- 
alog, as labeled. We see that at low redshifts, the fraction 
of redMaPPer clusters that is X-ray detected is clearly 
higher than that of the remaining cluster catalogs. By 
the z G [0.3,0.4] redshift bin — the AMF and redMaPPer 
algorithms have essentially identical performance, and at 
z G [0.4,0.5], the redMaPPer, AMF, and WHL catalogs 
all have comparable X-ray detection rates. 

These results clearly indicate that photometric noise 
and survey depth play a critical role in the performance 
of cluster finding algorithms. When the data is such that 
the photometric noise is low, redMaPPer clearly outper- 
forms the remaining cluster finding algorithms. As the 
cluster galaxies approach the magnitude limit of the sur- 



vey, however, the difference in performance between the 
various cluster catalogs becomes less pronounced. For fu- 
ture surveys like the DES or LSST, the redshift at which 
the photometric limit becomes comparable to the mag- 
nitude of the cluster galaxies being selected is expected 
to be z 0.9 and z«1.5 respectively. 

8.7. Centering 

As in our centering offset analysis of the redMaPPer 
catalog in Section [7J we match each of the photometric 
catalogs under consideration to an X-ray catalog com- 
prised of all unique galaxy clusters in the combined XCS, 
ACCEPT, and Mantz cluster catalogs. For each matched 
cluster, we compute the radial offset between the re- 
ported X-ray center and the reported optical center, from 
which we then compute the cumulative distribution func- 
tion. 

Our results are summarized in Figure 111! We see 
that the redMaPPer, maxBCG, gmBCG, and WHL clus- 
ter catalogs have essentially identical centering distribu- 
tions, while AMF performs significantly worse. Interest- 
ingly, the AMF algorithm is the only algorithm that does 
not rely on galaxy locations to define the cluster center. 
Rather, their cluster centers are based on maximizing a 
likelihood in the plane of the sky. Evidently, the prior 
that the cluster center falls on a cluster galaxy is highly 
informative, and leads to significant improvement in the 
centering performance. 

Just as interestingly, the redMaPPer, maxBCG, gm- 
BCG, and WHL algorithms vary widely in the sophis- 
tication of their centering algorithms, ranging from as 
simple as "pick the brightest member" , as done in WHL 
and maxBCG, to the iterative self-training algorithm of 
redMaPPer (see Paper I). On the one hand, this might 
suggest that the redMaPPer algorithm is unnecessarily 
complicated. On the other hand, our sophisticated al- 
gorithm has an important advantage relative to the re- 
maining cluster finders: redMaPPer is the only algorithm 
capable of estimating centering probabilities for each po- 
tential central galaxy. That said, we note that a detailed 
quantitative test of whether the redMaPPer centering 
probabilities are indeed correctly estimated has not yet 
been performed, a problem that we will return in a fu- 
ture work. For now, we will simply state that our in- 
ternal tests have produced good evidence that the cen- 
tering probability is well correlated with the likelihood 
of cluster being correctly centered, i.e. clusters with low 
centering probability are more often incorrectly centered. 
However, whether the centering probabilities are quanti- 
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Figure 9. Fraction of X-ray clusters detected by each of the optical catalogs, as a function of X-ray temperature (left) or X-ray luminosity 
(right) for all clusters with z G [0.1,0.5] (z G [0.1,0.3] for maxBCG). We caution this plot uses only an approximate treatment to determine 
which X-ray clusters fall within the optical masks of each catalog, which necessarily leads to an underestimate of the completeness. However, 
the treatment of the optical mask is the same across all catalogs (see text). 



tatively correct remains to be determined. 

9. SUMMARY AND CONCLUSIONS: 

We have performed an extensive quality test of the 
performance of the SDSS redMaPPcr cluster catalog by 
comparing it against several X-ray and SZ cluster cata- 
logs. We find that the redMaPPer redshifts are nearly 
unbiased, with biases falling at the 0.005 level or below. 
In addition, our photometric redshifts exhibit remark- 
ably low scatter, as low as sa 0.006 at low redshift, but 
increasing with redshift due to the increase in photo- 
metric noise in the data. Moreover, the scatter in red- 
shift is nearly symmetrical, and has a very low (~ 1%) 
catastrophic failure rate. In this context, it is important 
to emphasize that the word "catastrophic" here means 
that the true cluster redshift is more than 4er away from 
our photometric redshift estimate. Because our photo- z 
scatter is so low, clusters with \z spec — z p hoto| &s low as 
w 0.025 can be considered catastrophic failures. We note 
too that in addition to providing a simple photometric 
cluster redshift, the redMaPPer catalog includes an esti- 
mate of the full redshift probability distribution P(z) for 
every galaxy cluster (see Paper I). 

We have also tes ted the performance of our richness es- 
timator A (see also IRozo et al.l[2009bt IRvkoff et al.ll201^ 
using our reference X-ray catalogs. In particular, using 
the XCS and ACCEPT cluster catalogs we have esti- 
mated the scatter in X-ray temperature at fixed rich- 
ness, and converted it to an equivalent scatter in mass at 
fixed richness. Both data sets are consistent with each 
other, and the combined analysis suggest a scatter in 
mass at fixed richness of 26%. We have further verified 
this analysis by utilizing the Mantz cluster sample to 
measure the scatter in the Mg as -A relation, from which 
we are able to recover an equivalent mass scatter of 21%. 
Given the systematic uncertainties associated with our 
measurements — most importantly the fact that the se- 
lection function of the clusters with combined optical and 
X-ray data is not well constrained — we estimate that a 
reasonable range for the scatter in mass at fixed richness 



for the redMaPPer clusters is 25% ± 5%. Interestingly, 
this scatter may well be comparable to the estimated 
scatter from SZ ma ss proxies in curren t SZ surveys such 
as AC T and SPT (jBenson et all 120131: iHasselfield et al.l 
2013), and better than what ha s been achieved with 
Planck so far using SZ data only (jPlanck Collaboration! 
I2011bf) . We hasten to add, however, that we do expect 
significant improvement in future Planck data releases. 

In this context, it is also worth emphasizing that we 
did find one clear cluster outlier in our analysis — clus- 
ter 400D J 1416.4+4446 — whose richness was grossly 
over-estimated due to projection effects. The existence of 
this one outlier suggest that w 1% of our galaxy clusters 
suffer from projection effects. This failure rate is some- 
what lower than what was estimated in Paper I, where 
we found that up to « 5% of the galaxy clusters can be 
severely affected by projection effects, particularly at the 
rich end. The lower failure rate measured in this work 
is likely to be related to our reliance on external X-ray 
catalog for the majority of our tests, as X-ray selected 
sub-samples of the redMaPPer clusters are expected to 
be more robust to projection effects. 

We have also explored the completeness of the redMaP- 
Per cluster catalog as a function of X-ray luminosity, 
temperature, gas mass, and SZ detectability. redMaP- 
Per detects all galaxy clusters in the Mantz sample, AC- 
CEPT sample, and Planck ESZ sample. At low (z < 
0.35) redshifts, where the redMaPPer catalog is volume 
limited, the catalog is 100% complete at Tx > 3.5 keV, 
and above a luminosity threshold of Lx > 2 x 10 44 crgs/s. 
Because of the large scatter (w 60% — 70%) in the rela- 
tion between X-ray luminosity and cluster richness, the 
decrease in completeness with decreasing luminosity is 
very modest; at low redshifts, the catalog remains « 90% 
complete all the way down to Lx ~ 10 43 ergs/s. 

In addition, we have estimated the fraction of redMaP- 
Per galaxy clusters that are X-ray detected by cross- 
matching our catalog with the ROSAT Bright and Faint 
Source Catalogs. As expected, all rich, low redshift clus- 
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Figure 10. Fraction of optically selected galaxy clusters detected as X-ray sources in the ROSAT Bright or Faint Source Catalogs as a 
function of rcdshift, as labeled. X-ray sources are matched to clusters if they are within an angular separation 9 such that 8 subtends 
1 Mpc at the median cluster rcdshift within each bin. 



ters are X-ray detected, with the detection fraction de- 
creases with increasing redshift and/or decreasing rich- 
ness. We have verified that this decrease in the detection 
fraction is primarily — and most likely exclusively - 
driven by the decrease in the expected X-ray flux of the 
galaxy clusters with decreasing richness and increasing 
redshift. That is, we see no evidence of a large pop- 
ulation of intrinsically X-ray dark clusters. Of course, 
as noted above, we do have clear evidence for a popula- 
tion of galaxy clusters that suffers from projection effects, 
comprising w 1% of the cluster sample. 

We have further measured the miscentering rate of the 
redMaPPer galaxy clusters by visually inspecting 121 
galaxy clusters with high resolution X-ray data. Based 
on our visual inspection and the reported X-ray centers, 
we identified the correct central galaxy for each of these 
clusters, and compared it to the galaxy that was selected 
by the redMaPPer cluster catalog, finding that w 86% of 
the redMaPPer galaxy cluster are correctly centered. 

Finally, we also performed a "quick-and-dirty" version 
of all of the above analyses on all photometric cluster 
catalogs in the SDSS published to date. Our goal here is 



not to characterize in detail the performance of each of 
these cluster catalogs, but rather to provide a meaningful 
comparison among them. Note in particular that when 
performing these comparisons, the redMaPPer catalog is 
treated in the exact same way as the remaining cluster 
catalogs. 

As the result of this comparison, we find that the 
redMaPPer catalog has a higher completeness than that 
of any other cluster catalog. As for purity, we find that at 
low redshift the redMaPPer algorithm has a higher X-ray 
detection rate than that of any other catalog. However, 
as the survey depth becomes comparable to the mag- 
nitude of the bulk of the galaxy population of galaxy 
clusters, the differences between the various algorithms 
decrease. At z e [0.3,0.4], the redMaPPer and AMF 
algorithm have comparable X-ray detection rates, and 
at z £ [0.4,0.5], the WHL algorithm also reaches an X- 
ray detection rate comparable to that of redMaPPer and 
AMF. We note that in the Dark Energy Survey, the tran- 
sition point at which the survey depth becomes compa- 
rable to the magnitude of the cluster galaxies is z « 0.9. 

Turning to the redshift performance comparison, the 
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X-ray - Optical Offset (Mpc) 

Figure 11. Distribution of the radial offset between the X-ray 
and optical centers for matched galaxy clusters in each of the op- 
tical catalogs under consideration, as labeled. The X-ray sam- 
ple is comprised of all unique X-ray clusters from the combined 
XCS, ACCEPT, and Mantz cluster samples. All the catalogs have 
very similar centering performance characteristics except for AMF, 
which is the only catalog not to explicitly select a bright galaxy as 
the center. 



redMaPPer photometric redshifts are the least biased, 
with the WHL redshifts — themselves based on the DR8 
photometric redshifts — being equally unbiased. The 
redMaPPer redshifts clearly exhibit the lowest scatter 
amongst all catalogs, and, along with the gmBCG red- 
shift, they exhibit the least amount of skewness. The 
catastrophic failure rate of the redMaPPer redshifts is 
the lowest amongst the SDSS clusters catalogs. 

We also compared the performance of the redMaPPer 
richness estimator to that of the various other cluster 
catalogs, both via the Tx-richness scaling relations, and 
via the M gas -richness scaling relations. The redMaPPer 
catalog consistently exhibited the lowest scatter on all 
relations. Moreover, the redMaPPer catalog is the only 
one to consistently show evidence (> 3a) for the recov- 
ered scatter being lower than that estimated when the 
richnesses are randomly shuffled amongst all clusters. 

Finally, turning to cluster centering, we found that the 
centering algorithm in redMaPPer performs as well as 
— i.e., neither better nor worse — the remaining cata- 
logs. The only exception was the AMF catalog, which 
performed worse. Since AMF is the only catalog that 
did not choose to center the galaxy clusters by identify- 
ing a central galaxy, it is clear that the prior that galaxy 
clusters should be centered on bright cluster galaxies is 
highly informative, and improves the overall performance 
of a cluster finder. We also emphasize that while the 
remaining cluster catalogs all had equivalent centering 
performances, the redMaPPer catalog is unique in that 
the algorithm estimates the probability that the chosen 
central galaxy is correct, and also provides 4 additional 
candidate central galaxies along with their corresponding 
probabilities. Our hope is that cluster miscentering can 
be robustly treated using these estimated probabilities, 
but we postpone the development of a statistical frame- 
work that exploits this information to a future work. For 
now, we simply note that redMaPPer is the only algo- 



rithm for which such a framework can be developed. 

As an important caveat to our results, we wish to em- 
phasize that the results of our optical cluster comparison 
presented in this work reflect the reality of the various 
cluster finding algorithms as implemented in the SDSS, 
with the data that was available at the time. In partic- 
ular, it is worth keeping in mind that the performance 
of photometric redshift cluster finders such as AMF and 
WHL will depend not only on the cluster finding algo- 
rithm itself, but also on the quality of the redshifts em- 
ployed in the construction of the photo- z estimates. For 
instance, AMF relied on SPSS DR 6 data with photo-z 
estimates from lOvaizu et all (|2008l ). while WHL relied 
on SDSS DR8 photo-zs. Given that there has been a 
great deal of improveme nt between the DR6 photo-zs 
from lOvaizu et all (|2008f ) and the DR8 photo -zs — m 
large part due to increasingly large training samples for 
photo-z machine learning methods, with current sam- 
ples reaching over 850,000 galaxietQ — it is reasonable 
to expect the performance of AMF to improve if run us- 
ing DR8 photo-zs. That said, we believe that the fact 
that the redMaPPer performance is in no way limited 
by the performance of galaxy photo-z measurements is 
still clearly a valuable feature. Moreover, while redMaP- 
Per does require spectroscopic training samples, just as 
photo-z measurements do, it is important to emphasize 
that, as demonstrated in Paper I, redMaPPer can deliver 
photometric redshift quality using a spectroscopic train- 
ing sample comprised of only the 400 brightest cluster 
galaxies in SDSS (sampled over the full redshift range, 
see paper I for details). 

The excellent performance of redMaPPer in the Sloan 
Digital Sky Survey provides strong evidence that upcom- 
ing photometric surveys like the Dark Energy Survey and 
LSST will be capable of producing very large, high qual- 
ity cluster samples. Moreover, the excellent quality of 
the photometric redshifts, the low rate of catastrophic 
redshift failures and projection effects, and the low scat- 
ter in the optical mass proxy, all suggest that the quality 
of photometric cluster catalogs should easily suffice from 
the point of view of precision cosmology. In particular, 
a w 1% — 2% catastrophic failure rates on these algo- 
rithms is comparable to the expected precision of the 
cluster mass estimates that can be achieved in surveys 
like the Dark Energy Survey (e.g. lQguri fc Takadall2011l: 
iWeinberg et al.l r2012f ) . suggesting that optical detection 
systematics are not likely to be a dominant source of 
errors in near-future photometric surveys. The most im- 
portant exception remains cluster centering: with only 
85% of the redMaPPer galaxy clusters being correctly 
centered, there is a clear need for a robust statistical 
framework with which to address this problem, a ques- 
tion that we will return to in future work. 
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APPENDIX 

A. CLUSTERS WHERE MATCHINGS DIFFERED 
Below is the complete list of galaxy clusters where the cylindrical and membership matching algorithms differed. 

XCS: 

XCS J1310. 9+5720: No cylindrical match. The arc two clear galaxy clumps at this redshift, with the XCS cluster 
corresponding to the northern component. In rcdMaPPer, the southern component is richer, so it is this southern 



20 



Rozo & Rykoff 



component that was matched to the XCS cluster. However, the northern component is also detected, and is the obvious 
correct match. 

XCS J 0920. 8+3028: Cylindrical and membership matches differ. This cluster is at the eastern component of Abell 
781. rcdMaPPcr identifies both the eastern and western component, but the membership matching matches the XCS 
system to the western component because it is richer. 

XCS J0943. 5+1639: Cylindrical and membership matches differ. The membership match is correct. The cylindrical 
match for this clusters is a foreground cluster at z = 0.18 (compared to z = 0.25). 
XCS J2337. 9+2711: No cylindrical match. Membership match is clearly correct. 

XCS J0943.9+1641: No cylindrical match. Membership map is a larger cluster due NW which is not included in the 
XCS catalog. The correct match is found by rcdMaPPcr algorithm, but with a richness that falls below our selection 
threshold in Paper I. 

XCS J 1256. 8+2548: No cylindrical match. Membership match incorrect: correct match is a redMaPPer cluster that 
falls below our selection threshold. 

XCS J0840. 7+3830: No membership match. Correct match is clearly a redMaPPer cluster that falls below our 
selection threshold. 



MCXC: 

MCXC 31326.2+1230: No cylindrical match. This is a small satellite cluster of the much larger system Abell 1735, 
and does not pass the redMaPPer selection threshold. 

MCXC J 1230.7+3439: This is an interesting system. The X-ray location is close to a cD galaxy, which falls roughly 
in the middle of 2 redMaPPer clusters at the same redshift. The least rich of these 2 redMaPPer clusters is subject to 
masking from the richer system, which is the membership match of the X-ray system. In this sense, the membership 
match is clearly correct, but it may be subject to misccntering. Unambiguous evidence for miscentering would require 
high resolution X-ray imaging. 

MCXC J 1227. 1+1951: No cylindrical match. Membership match is clearly correct. 

MCXC J0751. 4+1730: No cylindrical match. The photometry around this region is clearly compromised. We set 
this cluster to unmatched to reflect the photometric failure, even though technically a cluster was found. 
MCXC J1415. 2-0030: No cylindrical match. This cluster is Abell 1882, which has two clear components. redMaPPer 
is centered on the NW component, whereas the MCXC system corresponds to the SE component. The SE component 
appears to be the most massive one based on the X-ray data, suggesting the cluster is catastrophically miscentered in 
redMaPPer. The membership matching association is clearly correct. 

MCXC J1311. 8+3227: No cylindrical match. Membership match is incorrect, with correct match falling below the 
redMaPPer selection threshold. 

MCXC J2258. 1+2055: Matchings differ. Membership match is clearly correct. 
MCXC J0943.5+1640: Matchings differ. Membership match is clearly correct. 
MCXC J1235. 1+4117: No cylindrical match. Membership match is clearly correct. 

MCXC J1337. 8+3854: No membership matching. X-ray clusters has no nearby bright galaxies, which is very unusual. 
We assume this cluster is unmatched, and note this may be an X-ray false detection. 

MCXC J1254. 8+255: No membership match. Correct match falls below rcdMaPPcr selection threshold, so cluster 
should be unmatched. 

MCXC J0943. 7+1644: Matchings differ. Membership match is clearly correct. 

MCXC J1436. 9+5507: No cylindrical match. Membership match is a larger foreground cluster. Cluster does not 

pass redMaPPer selection threshold, so it should be unmatched. 

MCXC J 1254-6+2545: No cylindrical match. Membership match is clearly correct. 

There are no clusters where the matchings differed in the ACCEPT, Mantz, and Planck reference cluster catalogs. 



B. NOTES ON UNMATCHED CLUSTERS 
Below is the complete list of unmatched clusters in our reference catalogs. 



XCS: 

XCS J0943.9+1641: The cluster was assigned a high richness because it neighbors a richer system that is not 
included in XCS. The cluster should be unmatched. 



MCXC: 

MCXC J0751. 4+1730: This cluster was identified as a catastrophic photometric failure, and was therefore unmatched 
by hand in our visual inspection. 

MCXC J0159. 3+0030: This is a very rich cluster, but gets masked out from redMaPPer because at its optical center, 
more than 20% of the cluster is lost to the galaxy mask. In other words, the cluster doesn't formally fall within the 
angular redMaPPer selection region. 

MCXC J0159. 3+0030: Like the previous cluster, this system is lost to the galaxy mask since at its optical center, more 
than 20% of the cluster is lost to the galaxy mask. Thus, the cluster is formally not within the angular redMaPPer 
selection region. 
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There are also 4 clusters that are matched to redMaPPer systems, but fall below our selection threshold at the 
redMaPPer center/redshift. These are MCXC J0927.1+5327, MCXC J1340.9+3958, J1011.0+5339, and J1334.5+3756. 

Planck ESZ: 

PLCKG96.9+52.5: Like the previous cluster, this system is lost to the galaxy mask since at its optical center, more 
than 20% of the cluster is lost to the galaxy mask. Thus, the cluster is formally not within the angular redMaPPer 
selection region. 

There are no unmatched clusters in the ACCEPT and Mantz reference catalogs. 

C. NOTES ON REDSHIFT OUTLIERS 
Below is the complete list of redshift outliers between all reference cluster catalogs. 

XCS Outliers: 

XMMXCS J 0921. 2+3701: Cluster is miscentered in optical based on X-ray data. The redMaPPer cluster center has 
a spectroscopic redshift z sp cc = 0.235, in agreement with the redMaPPer photometric redshift. It seems likely that 
the reference redshift is incorrect, but a spectroscopic redshift of the correct central galaxy is unavailable. 



SDSS spectra confirm redMaPPer redshift. 
SDSS spectra confirm redMaPPer redshift. 
SDSS spectra confirm redMaPPer redshift. 
SDSS spectra confirms redMaPPer redshift. 
SDSS spectra confirm redMaPPer redshift. 

SDSS spectra unavailable, but visual inspection strongly suggests reference redshift is incor- 



MCXC Outliers: 

MCXC J 1621. 0+2546: 
MCXC J 1421. 6+3717: 
MCXC J 1621. 0+2546: 
MCXC J 1017. 5+5934: 
MCXC J0935.4+0729: 
MCXC 32135. 2+0125: 
rect. 

MCXC 30809.6+2811: redMaPPer redshift compromised by bad SDSS photometry from a nearby star. 
MCXC 30826.1+2625: Our photometric redshifts at the redMaPPer center and the reference cluster center disagree. 
The cluster is properly centered by redMaPPer, and the corresponding photometric redshift is correct based on that 
galaxy's spectroscopic redshift. 

MCXC 30847.1+3449: Optical inspection reveals the redMaPPer match is in fact a foreground cluster in the vicinity 
of the X-ray cluster. The correct cluster match is detected by redMaPPer, but falls below the richness threshold, likely 
due to masking by the foreground cluster. This cluster should be formally unmatched. We update our matchings 
appropriately, and remove this system from the list of matched clusters. 

MCXC 31011. 4+ 54 50: The X-ray center falls between two galaxy clumps. Both clumps are the same photometric 
redshift z\ ps 0.35, suggesting that the reference redshift z rc f = 0.294 is incorrect. 
MCXC 30943.1+4659: SDSS spectra confirm MCXC redshift. 
MCXC 30124.5+0400: SDSS spectra confirm MCXC redshift. 

MCXC 31447-4+0827: This cluster has a spectacular star bursting galaxy at its center, with z spcc = 0.375, 
confirming the redMaPPer redshift. 

ACCEPT Outliers: 

MACS 32211.7-0349: No SDSS spectra, but redshift value in lMantz et~aTl (|2010bf ) confirms the redMaPPer redshift. 
AMI 1763: SDSS spectra confirm redMaPPer redshift. 

Mantz Outliers: 

400d 30809.6+2811: This cluster is the same as MCXC J0809.6+2811 (above). The redMaPPer redshift is compro- 
mised by bad SDSS photometry from a nearby star. 

Planck ESZ Outliers: 

PLCKG56. 0-34-9: This cluster is the same as MCXC J2135. 2+0125. Spectra are unavailable, but visual inspection 
strongly suggests the MCXC redshift is incorrect. 

D. NOTES FROM CENTERING ANALYSIS 

XCS: 

XCS 32239.4-0547: This is one of a pair of galaxy clusters at the same redshift, the second being XCS J2239. 7-0543. 
Each XCS system has Tx = 2.8, but the pair is identified as a single redMaPPer cluster. Since even the X-ray center 
is ambiguous, we remove these cluster from the centering analysis. 

XMMXCS 31052.4+4419: There are two clusters near this location at spectroscopic redshift z = 0.44 and z = 0.50. 
The latter is the XCS system. redMaPPer finds both clusters, but the XCS system is heavily masked by the lower 
redshift objects, and falls below the detection threshold, so it is difficult to determine whether this system is a 
centering failure or not. We remove the system from our centering study. 
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Figure 12. Distribution of the redshift offset z spec — z v ]^ oto for the [Wen ct al. (2012) catalog, measured using rcdshifts exclusive to DR9 
(solid black histogram) and redshifts from the DR8 release (red histogram). All clusters in the redshift range z p hoto G [0.1,0.5] arc included. 
The distribution of the DR8 redshift offsets is clearly artificially truncated at |Az| < 0.055, thereby removing the tails of the distribution. 

ACCEPT: 

Abell 115 (3C 28.0): The cluster has a N and S component. rcdMaPPer is centered on the S component, b ut ACCEPT 
center s the cluster on the N comp onent, which dominat es the X-ray em i ssion. However, both weak lensing lOkabe et al.l 
(|2010f ) : IAMI Consortium! ()2011[ ) and dynamical data iBarrena et al.l (|2007[ ) reveal 2 addit ional substructure s both 
support the S component being the dominant one, by a factor of sa 4 — 10 in mass, which led IAMI Consortium! ([201 ID 
to suggest that the X-ray emission from the N component is in fact dominated by emission associated with the ratio 
source 3C 28. The S component is also clearly dominant in the optical. Consequently, we assume this clusters was 
correctly centered, but caution that our conclusion may be incorrect due to correlated scatter between optical richness, 

weak lensing mass, and velocity dispersion. 

Abell 1758: A1758 has been extensively studied (|David fc Kempnerl[200l lOkabe fc Umetsull2008t lHaines et aJ]l2009t 
iDurret et alj|20lfaj) . It is typically split into two components — A1758N and A1758S — separated by sa 2 Mpc. Both 
components are identified as independent clusters by redMaPPer, but it is only A1758N which is included in ACCEPT. 
A1758N is itself a merging system, with X-ray and optical data suggesting that the N component dominates, implying 
redMaPPer correctly identified the central galaxy of this cluster. Our central galaxy coincides with the X-ray peak, 

but is significantly offset from the X-ray centroid. 

Abell 1914: This is a c omplicated merging system with a highly irregular mass distribution (Okabc fc Umetsul 120081 : 
IAMI Consortium! 120 111) . Both X-ray and SZ data suggest the brightest cluster galaxy is the center of the dominant 
componen t, though the weak lensi ng k peak of the NE component appears to be slightly higher (but less extended). 
We follow lOkabe fc Um ctsu (200l|) and tentatively associate the SW component as the dominant clump, which makes 
Abell 1914 a redMaPPer centering success. 

Abell 370: There are two comparably bright cD galaxies near the X-ray center of almost equal brightness (Am = 0.05). 
Contrary to the other times when we were faced with a similar decision, in this case we assigned the dimmer galaxy as 
the correct center, based on the curvature of an obvious giant arc. Our visual choice agrees with the redMaPPer center. 

Mantz: 

MACS J2311. 5+0338: Cluster has distinct NE and SW components. redMaPPer centered on the NE, but X-rays 
indicate that the SW component is dominant. 

E. EVIDENCE FOR SPECTROSCOPIC CUTS IN THE WHL CATALOG 

As noted in section 187X1 we evaluate the redshift performance of the WHL catalog using clusters with spectroscopic 
redshifts exclusive to DR9. Our motivation for excluding all DR8 photo-zs from consideration is shown in Figure [T2l 
where we compare the distribution of the redshift offset z spec — z p ] loto as evaluated using DR8 spectra (red dashed 
histogram), to that obtained using rcdshifts exclusive to D R9 (solid black h istogram). We note that the redshifts 
exclusive to DR9 were not publicly available at the time the iWen et ail (|2012[ ) catalog was published. It is clear from 
the figure that the distribution for the DR8 sub-sample is artificially truncated at \z spec — z p hoto\ < 0.055. Given that 
there is clear evidence this sub-sample of galaxy clusters was subject to a spectroscopic redshift cut, we evaluate the 
redshift performance of the WHL algorithm using only clusters with spectroscopy exclusive to DR9. 



